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1  Introduction 


The  Improved  Concurrent  Electromagnetic  Particle  in  Cell  (ICEPIC)  code  is  an  advanced,  parallel 
simulation  tool  for  high  power  microwave  (HPM)  sources  [  1]— [3] .  ICEPIC  is  an  electromag¬ 
netic  particle  in  cell  (EM-PIC)  code  that  most  often  employs  a  regular  Cartesian  grid  of  cells  to 
discretize  the  computational  space;  cylindrical  coordinate  grids  are  also  available,  but  are  not  dis¬ 
cussed  herein.  The  development  of  the  code  at  the  High  Power  Microwave  Division,  Directed 
Energy  Directorate,  Air  Force  Research  Laboratory  (AFRL/DEH)  is  mature  to  the  point  that  the 
code  is  used  to  design  and  improve  devices  before  these  devices  are  constructed  and  tested  in  the 
laboratory. 

One  such  device  that  is  simulated  before  its  construction  is  an  A6  relativistic  magnetron.  Fol¬ 
lowing  ICEPIC  simulation  and  design  improvement,  actual  construction  and  testing  of  the  device 
shows  that  most  of  the  ICEPIC  predictions  are  physically  accurate.  However,  ICEPIC  under  pre¬ 
dicts  the  input  impedance  of  the  device.  This  under  prediction  runs  counter  to  experience  and 
intuition  because  laboratory  devices  generally  exhibit  parasitic  current  losses  that  are  not  modeled 
in  numerical  simulation  codes.  The  parasitic  current  losses  cause  the  laboratory  device  to  draw 
more  current  than  the  simulated  device;  hence,  the  input  impedance  is  usually  over  predicted  in 
numerical  simulation. 

Due  to  the  under  prediction  of  the  input  impedance,  questions  are  raised  about  the  size  of  the 
grid  cells  in  the  ICEPIC  simulations.  A  numerical  convergence  study  of  the  device  is  undertaken 
to  resolve  these  questions.  The  results  of  the  study  provide  guidance  for  future  A6  magnetron 
simulation  and  are  more  broadly  applicable  to  the  simulation  of  similar  HPM  devices. 

2  Simulation  background 

The  ICEPIC  simulations  leading  to  the  construction  of  the  relativistic  magnetron  utilize  2mm  cubic 
cells.  At  the  expected  /,-band  frequency  of  oscillation,  this  is  more  than  sufficient  to  resolve  the 
electromagnetics  in  the  problem.  However,  it  allows  for  only  9  cells  across  the  anode-cathode 
(AK)  gap,  and  even  fewer  cells  across  the  expected  electron  hub  height.  These  features  may  be 
under  resolved  in  the  simulations. 

To  gauge  the  convergence  of  the  simulations,  simulations  are  run  using  2mm  cells,  1mm  cells, 
and  0.5mm  cells.  If  the  1mm  results  are  significantly  different  from  the  2mm  results,  the  2mm 
results  are  likely  unconverged.  Similarly,  if  the  0.5mm  results  differ  significantly  from  the  1mm 
results,  the  1mm  results  are  likely  unconverged.  On  a  COMPAQ  ES-45  computer,  the  observed 
computational  burden  of  the  2mm  simulations  is  approximately  2.5  hours  on  16  processors;  the 
burden  of  the  1mm  simulations  is  approximately  20.5  hours  on  32  processors,  and  the  burden 
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of  the  0.5mm  simulations  is  approximately  86  hours  on  128  processors.  This  places  the  0.5mm 
simulations  near  the  limit  of  currently  available  computational  resources.  Note  that  cutting  the  cell 
size  in  half  increases  the  number  of  simulation  cells  by  a  factor  of  8  and  requires  the  simulation 
timestep  to  be  cut  in  half.  Thus,  the  expected  increase  in  computational  resources  is  a  factor  of  16; 
the  observed  increases  are  close  to  this  expected  value. 

ICEPIC  uses  macro  particles  during  a  simulation,  and  each  macro  particle  represents  many 
electrons.  At  each  timestep,  a  cell  on  the  cathode  emits  a  fixed  number  of  macro  particles,  and  the 
appropriate  charge  according  to  physical  field  emission  laws  is  assigned  to  each  particle.  Increasing 
the  number  of  macro  particles  emitted  increases  the  simulation  fidelity.  To  gain  more  insight  into 
the  convergence  properties,  each  simulation  is  run  with  2  emitted  particles  per  cell  per  timestep 
and  8  emitted  particles  per  cell  per  timestep. 

During  the  course  of  the  magnetron  development,  two  major  modifications  are  tested.  One 
modification  is  to  the  anode  geometry  and  one  is  to  the  power  extraction  scheme.  To  gain  further 
insight,  three  different  configurations  of  the  magnetron  are  studied.  The  first  configuration  employs 
the  modified  anode  geometry  and  the  original  extraction  scheme.  The  second  employs  the  original 
anode  geometry  but  the  modified  extraction  scheme.  The  third  and  final  configuration  employs  the 
modified  anode  geometry  and  the  modified  extraction  scheme. 

In  total,  18  simulations  are  studied  (3  grid  resolutions  x  2  emission  settings  x  3  magnetron 
configurations  =  18  simulations.)  Other  simulation  parameters  such  as  the  input  voltage  and  ap¬ 
plied  magnetic  field  are  held  constant  across  all  of  the  simulations.  Data  from  all  of  these  simula¬ 
tions  is  analyzed  for  convergence  trends.  There  is  a  large  number  of  predicted  quantities  that  can 
be  examined  from  these  simulations.  Three  of  these,  the  steady  state  input  impedance,  the  average 
output  power,  and  the  frequency  of  oscillation  are  chosen  as  representative  and  are  the  focus  of 
this  report.  In  examining  these  quantities,  the  focus  is  on  how  the  quantities  change  as  the  grid  cell 
size  and  the  number  of  emitted  particles  is  changed  rather  than  on  the  absolute  magnitude  of  the 
quantities. 

3  Convergence  criteria 

Even  before  performing  the  18  simulations  discussed  above  in  Sec.  2,  an  initial  convergence  scan 
of  the  modified  anode,  original  extraction  scheme  magnetron  is  performed  at  2  emitted  particles 
per  cathode  cell  per  timestep.  A  time-windowed  average  of  the  output  power  from  one  of  the 
extractors  at  the  three  grid  resolutions  is  shown  in  Fig.  1.  Alarmingly,  these  results  appear  to  be  not 
converging.  The  change  in  steady  state  average  power  from  the  1mm  results  to  the  0.5mm  results 
is  approximately  the  same  magnitude  as  the  change  from  the  2mm  results  to  the  1mm  results. 
Further,  the  device  startup  time  decreased  from  the  2mm  simulation  to  the  1mm  simulation,  then 
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Figure  1:  Time-windowed  average  power  from  one  extractor  from  the  initial  convergence  scan. 
The  results  do  not  show  convergence. 

Table  1:  Summary  of  how  steady  state  results  change  with  cell  size  in  the  initial  convergence  scan. 
The  simulation  appears  not  to  be  converging. 


Cell  Size 

Output  Power 

Input  Impedance 

Frequency 

2mm  to  1mm 

4.92% 

4.09% 

0.00% 

1mm  to  0.5mm 

4.09% 

2.56% 

0.00% 

increased  from  the  1mm  simulation  to  the  0.5mm  simulation.  This  indicates  serious  problems  with 
the  simulation.  Other  quantities  from  this  initial  convergence  scan  are  summarized  in  Table  1. 

Upon  inspection,  during  this  initial  scan,  the  magnetron  is  found  not  to  be  centered  on  the 
computation  grid,  or,  in  other  words,  the  center  axis  of  the  magnetron  runs  through  the  grid  at  an 
arbitrary  location  within  a  grid  cell  rather  than  coinciding  with  a  cell  corner  or  the  center  of  a  cell. 
This  causes  the  magnetron  to  be  meshed  asymmetrically,  as  shown  in  Fig.  2. 

Fortunately,  when  this  condition  is  corrected  by  centering  the  magnetron  so  that  its  axis  coin¬ 
cides  with  a  grid  cell  comer,  the  grid  exhibits  top  to  bottom  and  left  to  right  symmetry  as  shown 
in  Fig.  3.  This  centering  condition  is  obeyed  for  all  results  subsequently  discussed  herein.  These 
results  show  that  centering  the  magnetron  on  the  computation  grid  does  lead  to  numerical  conver¬ 
gence.  Thus,  the  simulation  geometry  should  be  appropriately  centered  on  the  computation  grid  in 
general  for  EM-PIC  simulations  such  as  those  performed  by  ICEPIC  to  be  expected  to  converge. 


4  Simulation  data 

Plots  of  the  normalized  output  power  of  the  various  magnetron  configurations  as  a  function  of  time 
are  shown  in  Figs.  4-6.  As  discussed  in  Sec.  3,  the  magnetron  is  centered  on  the  grid  for  all  of 
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Figure  2:  Closeup  of  the  magnetron  cathode  with  the  magnetron  axis  at  an  arbitrary  grid  location. 
Note  the  top  to  bottom  asymmetry. 


Figure  3:  Closeup  of  the  magnetron  cathode  with  the  magnetron  axis  at  a  grid  cell  corner.  Note  the 
top  to  bottom  symmetry. 


(a)  2  emitted  particles  per  cell  per  timestep  (b)  8  emitted  particles  per  cell  per  timestep 

Figure  4:  Time- windowed  average  power  from  one  extractor  from  modified  anode,  original  extrac 
tion  magnetron. 


(a)  2  emitted  particles  per  cell  per  timestep  (b)  8  emitted  particles  per  cell  per  timestep 

Figure  5:  Time- windowed  average  power  from  one  extractor  from  original  anode,  modified  extrac 
tion  magnetron. 
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(a)  2  emitted  particles  per  cell  per  timestep  (b)  8  emitted  particles  per  cell  per  timestep 


Figure  6:  Time-windowed  average  power  from  one  extractor  from  modified  anode,  modified  ex¬ 
traction  magnetron. 


these  simulations.  Note  that  a  few  simulations  ended  prematurely  because  the  computer  platform 
allowed  simulations  to  run  for  a  maximum  of  96  hours.  The  simulations  shown  in  Figs.  4-6 
appear  to  be  converging  much  better  than  the  simulation  shown  in  Fig.  1,  where  the  magnetron 
is  not  centered  on  the  grid.  In  these  latter  simulations,  the  change  in  steady  state  output  power  is 
smaller  when  the  grid  is  refined  from  1mm  cells  to  0.5mm  cells  than  it  is  when  the  grid  is  refined 
from  2mm  cells  to  1mm  cells.  Also,  with  the  possible  exception  of  the  original  anode,  modified 
extraction  configuration,  which  is  known  to  have  physical  stability  problems,  the  change  in  the 
time  to  reach  steady  state  does  not  decrease  then  increase  again  with  grid  refinement  as  it  does  in 
Fig.  1.  The  convergence  situation  appears  to  be  much  improved  by  centering  the  magnetron  on  the 
computation  grid. 

The  changes  in  simulation  quantities  as  the  grid  cell  size  is  changed  are  summarized  in  Table  2, 
and  the  changes  as  the  number  of  emitted  particles  per  cell  per  timestep  is  varied  are  summarized 
in  Table  3.  In  comparing  the  two  tables  it  is  apparent  that  changing  from  2  emitted  particles  per 
cell  per  timestep  to  8  particles  per  cell  per  timestep  has  a  much  smaller  effect  on  the  convergence 
of  the  simulation  than  changing  the  cell  size.  Therefore,  2  emitted  particles  per  cell  per  timestep 
is  deemed  adequate  for  these  simulations.  The  output  power,  input  impedance,  and  frequency  of 
oscillation  of  the  magnetron  change  by  comparatively  large  amounts  as  the  cell  size  is  varied  from 
2mm  to  1mm,  but  these  quantities  change  by  much  smaller  amounts  as  the  cell  size  is  varied  from 
1mm  to  0.5mm.  Thus,  1mm  simulation  should  be  used  for  most  design  work.  A  0.5mm  simulation 
is  advisable  to  check  a  design  before  committing  resources  to  construction  of  an  actual  magnetron. 
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Table  2:  Summary  of  how  steady  state  results  change  with  cell  size  for  the  various  simulations. 


Configuration 

Emission 

Cell  Size 

Power 

Input  Impedance 

Frequency 

Mod.  anode, 
orig.  ext. 

2  parts/cell/step 

2mm  to  1mm 

2.31% 

6.88% 

2.70% 

1mm  to  0.5mm 

1.80% 

1.31% 

0.00% 

8  parts/cell/step 

2mm  to  1mm 

3.63% 

6.69% 

2.70% 

1mm  to  0.5mm 

0.92% 

1.27% 

0.00% 

Orig.  anode, 
mod.  ext. 

2  parts/cell/step 

2mm  to  1mm 

8.45% 

5.23% 

0.00% 

1mm  to  0.5mm 

0.93% 

3.71% 

0.00% 

8  parts/cell/step 

2mm  to  1mm 

7.46% 

5.59% 

0.00% 

1mm  to  0.5mm 

4.39%* 

3.77% 

0.00% 

Mod.  anode, 
mod.  ext. 

2  parts/cell/step 

2mm  to  1mm 

4.41% 

7.88% 

2.94% 

1mm  to  0.5mm 

2.76% 

1.55% 

0.00% 

8  parts/cell/step 

2mm  to  1mm 

3.09% 

7.97% 

2.94% 

1mm  to  0.5mm 

2.10% 

2.44% 

0.00% 

* 


Value  may 


be  inaccurate  due  to  truncation  of  simulation  by  hard  wall  clock  time  limit. 


Table  3:  Summary  of  how  steady  state  results  change  with  particle  emission  for  the  various  simu¬ 
lations. 


Configuration 

Cell  Size 

Emission 

Power 

Input  Impedance 

Frequency 

Mod.  anode, 
orig.  ext. 

2mm 

2  to  8  parts/cell/timestep 

-0.38  % 

0.19% 

0.00% 

1mm 

2  to  8  parts/cell/timestep 

0.91% 

0.01% 

0.00% 

0.5mm 

2  to  8  parts/cell/timestep 

0.03% 

-0.03% 

0.00% 

Orig.  anode, 
mod.  ext. 

2mm 

2  to  8  parts/cell/timestep 

-0.05  % 

0.08% 

0.00% 

1mm 

2  to  8  parts/cell/timestep 

-0.96% 

0.42% 

0.00% 

0.5mm 

2  to  8  parts/cell/timestep 

2.44%* 

0.48% 

0.00% 

Mod.  anode, 
mod.  ext. 

2mm 

2  to  8  parts/cell/timestep 

0.10% 

0.03% 

0.00% 

1mm 

2  to  8  parts/cell/timestep 

-1.16% 

0.12% 

0.00% 

0.5mm 

2  to  8  parts/cell/timestep 

-1.79% 

0.99% 

0.00% 

* 


Value  may 


be  inaccurate  due  to  truncation  of  simulation  by  hard  wall  clock  time  limit. 


5  Simulation  run  times 


Table  4  summarizes  the  number  of  processors  and  the  run  times  for  the  various  simulations  on  a 
COMPAQ  ES45  computer.  The  0.5mm  cell  size  simulations  are  close  to  the  limit  of  computational 
resources  on  current  super  computers.  In  fact,  some  of  these  simulations  that  employed  8  particle 
per  cell  per  timestep  emission  did  not  run  to  completion  in  the  96  hours  allowed  on  the  computer. 
The  next  generation  of  super  computers  may  allow  for  a  0.25mm  cell  size  run.  Note  that  the 
expected  increase  in  CPU  time  (wall  clock  time  times  number  of  processors)  when  halving  the  cell 
size  is  a  factor  of  16  as  the  number  of  cells  increases  by  a  factor  of  2  in  each  of  3  dimensions  and 
the  maximum  stable  timestep  is  cut  in  half,  which  doubles  the  number  of  required  timesteps.  The 
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Table  4:  Summary  of  simulation  run  times  on  a  COMPAQ  ES45  computer.  Times  are  hh:mm:ss. 
Some  simulations  did  not  run  to  completion  due  to  a  96  hour  run  time  limit. 


Configuration 

Emission 

Cell  Size 

Processors 

Wall  Time 

Mod.  anode, 
orig.  ext. 

2  parts/cell/step 

2mm 

16 

2:02:35 

1mm 

32 

15:38:32 

0.5mm 

128 

70:56:05 

8  parts/cell/step 

2mm 

16 

2:54:31 

1mm 

32 

18:15:59 

0.5mm 

128 

96:00:00+ 

Orig.  anode, 
mod.  ext. 

2  parts/cell/step 

2mm 

16 

2:38:22 

1mm 

32 

20:12:30 

0.5mm 

128 

96:00:00+ 

8  parts/cell/step 

2mm 

16 

3:27:44 

1mm 

32 

23:20:02 

0.5mm 

128 

96:00:00+ 

Mod.  anode, 
mod.  ext. 

2  parts/cell/step 

2mm 

16 

2:34:48 

1mm 

32 

20:28:21 

0.5mm 

128 

86:07:16 

8  parts/cell/step 

2mm 

16 

3:23:32 

1mm 

32 

25:31:56 

0.5mm 

128 

96:00:00+ 

data  in  Table  4  reflects  this  expectation.  By  comparison,  the  increase  in  CPU  time  due  to  increasing 
from  2  particles  per  cell  per  timestep  to  8  particles  per  cell  per  timestep  is  modest,  but  the  data  in 
Sec.  4  shows  little  or  no  convergence  gain  by  doing  so. 


6  Conclusion 

The  convergence  properties  of  ICEPIC  magnetron  simulations  are  examined  experimentally.  Three 
different  magnetron  configurations  are  examined  each  at  three  different  grid  cell  sizes  and  two 
different  particle  emission  settings.  The  simulations  reveal  that  the  magnetron  must  be  centered 
on  the  computational  grid  for  simulation  convergence.  Further,  the  predicted  input  impedance 
of  the  magnetron  increases  as  the  computational  grid  is  refined,  bringing  the  value  closer  to  the 
observed  value  in  the  laboratory.  The  particle  emission  setting  is  found  to  have  little  effect  on  the 
convergence.  Simulations  employing  2mm  cell  size  are  found  to  be  unconverged;  1mm  cell  size  is 
adequate  for  most  design  work,  and  a  0.5mm  cell  size  check  is  advisable  before  finalizing  a  design. 
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